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Abstract 


Between 2014 and 2017, an interdisciplinary group of researchers examined whether a high 
school health class that incorporated mindfulness techniques lowered students’ stress and 
improved their wellbeing. We received a grant from the Institute of Educational Sciences from the 
United States Department of Education to (1) train high school health teachers to implement the 
mindfulness-based curriculum and (2) study how the program impacted students. We examined 
changes in cognitive, social, emotional, and physical wellbeing indicators from before the course 
to after the course, using a quasi-experimental (intervention and control group with pre and post 
tests), mixed-methods (concurrent triangulation) design. Data consisted of student responses to 
psychometric surveys and interviews with a subsample of students. When we compared 
qualitative findings with quantitative outcomes, we discovered instances of different results 
through each method. This allowed us to provide explanations and context for certain outcomes. 
For example, the recent loss of a grandparent explained why one student scored high in social 
isolation. At other times, the two methods found two different “outcomes stories” from the same 
student. For example, a student reported in the interview that he was struggling with social 
interactions, but the psychometric indicators had him in the top quintile of social wellbeing 
indicators as compared to his peers. This case study discusses methodological approaches to 
mixed-method educational psychology research, the challenges of making sense of data from 
different sources, and the usefulness and limitations of mixed-methodologies for psychological 
outcome evaluations of student wellbeing interventions. We also discuss the process of working 
as a large group of researchers. This case is useful for students in psychology and education, 
and it is equally useful for early career researchers. 


Learning Outcomes 


By the end of this case, students should be able to 


e State advantages and drawbacks of different mixed-methods approaches for psychological 
and educational research 

e Identify methodological approaches that lend themselves to particular kinds of research 
findings 

e Evaluate how to make sense of divergent findings from different methods in mixed-method 
studies 


Project Overview and Context 


The research took place between 2014 and 2017. Data collection and the first stage of analysis 
have been completed. The preliminary results have been presented at national and international 
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conferences, and research papers are in preparation. The research team consisted of 
established scholars in psychology and education, early- and mid-career researcher associates, 
and advanced doctoral students. The team was organized into two main sub-groups, qualitative 
and quantitative, with the principal investigator coordinating with each. The work process is 
discussed in more depth below. 


The research was designed to study the impact of an educational intervention. Health teachers 
were trained to present a 12-lesson mindfulness-based wellbeing program (12, 45-min lessons) 
called Learning to BREATHE or L2B. The L2B curriculum focuses on cultivating students’ 
awareness of their thoughts, feelings, and physical sensations to improve their development of 
emotion regulation and attentional skills. 


We performed this research in an Eastern state in the United States at two public, suburban high 
schools outside of a midsized city. Overall, 385 adolescents aged 15 to 17 participated in 
answering a battery of surveys and playing cognitive “games” to measure their emotional, social, 
and physical wellbeing as well as changes in cognitive functions. A subsample of the students 
was also interviewed to gain a deeper understanding of their experience, certain aspects of the 
L2B curriculum, and self-defined changes from the students’ perspectives. The student 
population at both schools was ethnically diverse. School A had 23% students of color from 
various backgrounds (Latino, Black, Asian, first and second generation immigrants), while School 
B consisted of 62% students of color. They were also socio-economically diverse (14% of 
students on free/reduced lunch at School A, and 47% at School B), and had a range of academic 
aptitudes. 


The research design was mixed-method and quasi-experimental, meaning that we collected both 
qualitative (interview) data and quantitative (Survey and cognitive “game”) data, and that we 
studied the intervention (the L2B course) in comparison to a control group who did not did 
receive the L2B course during the study (Creswell, 2013; Creswell & Clark, 2007; Creswell, Clark, 
Gutmann & Hanson, 2003; Maxwell & Loomis, 2003). In each school, one group of students (n = 
132) participated in the L2B curriculum, and a control group (n = 119) participated in the standard 
district health course curriculum for a total of 251 participants in the randomized controlled trial 
(134 student participated in the pilot study). Students in both groups at both schools were tested 
before the intervention and after the intervention to compare scores and measure change 
between pre-course scores and post-course scores. The students were also interviewed after the 
intervention to reflect upon what they learned and how the program impacted them. 


Gaining access to these schools and administering the surveys posed some challenges, but the 
greatest challenges involved interviewing the students, which will be discussed in-depth below. 
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The research questions for this aspect of the project were as follows: 


1.How do high school students experience and manage stress in their lives? 
2.What are the effects, if any, of a mindfulness curriculum on students’ wellbeing? 


Research has shown that high school aged students face significant stress relative to other 
stages of development (Tottenham & Galvan, 2016), which has been attributed to increases in 
school dropout (Kessler, Foster, Saunders, & Stang, 1995), as well as negative effects on 
student’s grades, healthy decision-making, cognitive functioning, and overall wellbeing (Gumora 
& Arsenio, 2002; Sapolsky, 2003; Vermeersch, T’Sjoen, Kaufman, Vincke, & Bracke, 2010). With 
the concerning rates of depression and school drop out, educational psychologists and policy- 
makers have begun to pay greater attention to how schools can teach students approaches to 
lower their stress and cultivate student wellbeing to foster healthy attitudes and behaviors. One 
approach to fostering student wellbeing is through health courses that integrate mindfulness 
techniques to allow students to de-stress while in class and to learn about effects of stress on 
wellbeing and decision-making (Broderick & Jennings, 2012). Research on mindfulness for 
adults has shown positive outcomes for stress reduction and wellbeing (Chiesa & Serretti, 2009; 
Frank, Reibel, Broderick, Cantrell, & Metz, 2013), which is why this research project set out to 
understand the effects of mindfulness practices on adolescents, and the possibility of 
incorporating the techniques in school. 


This study is unique in using a quasi-experimental and mixed-methods research design to 
examine a mindfulness-based intervention facilitated by a classroom teacher in a universal health 
class. In fact, most research projects on mindfulness and psychological wellbeing rely on 
quantitative, self-report measures, even though scholars have found limitations to this approach, 
such as work from Davidson and Kasniack (2015). Heeding this call, our research employed a 
specific mixed-methodology to better triangulate findings and draw a more detailed picture of the 
effect of the interventions on students. 


Research Practicalities 


The research was made possible thanks to a large multi-year grant from the Institute of 
Educational Sciences (U.S. Dept. of Education). Mark Greenberg and Jennifer (Jenny) Frank 
from Penn State University were the co-principal investigators (Pls), while a team of quantitative 
and qualitative researchers performed particular data collection and data analysis tasks. The 
qualitative team consisted of Deborah Schussler, Julia Mahfouz, and Joseph Levitan, and the 
quantitative team consisted of Joy Mitra, Yoonkyung Oh, and Kimberly Kohler. Elaine Berrena 
and Patricia (Trish) Broderick focused on teacher training for implementation of the curriculum, 
and also developed evaluation protocols for implementation fidelity. Trish Broderick (2013) is also 
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the author of the L2B curriculum. Each team was fairly autonomous in their respective tasks. This 
rather large interdisciplinary team spanned the Departments of Educational Psychology, Human 
Development and Family Studies, Education Leadership, and Special Education. 


The large size of the full research team allowed for in-depth data collection from both qualitative 
and quantitative methodologies but also made communication and setting meeting times a 
challenge. Each smaller team met separately on a regular basis, and the full team would meet 
bi-weekly, on average, depending upon the phase of the research project. The large size of the 
full research team provided depth and expertise for each phase of the research, which was a 
considerable strength. Despite this strength, there were a number of challenges to overcome in 
order to complete the research, which we discuss below. 


Sampling, Teacher Recruitment, and Access to Schools 


Establishing a relationship with schools willing to partner and work to achieve the fundamental 
goals of the program, and participate fully in the research, is a necessity for any successful 
project. Regular visits to the schools were required for a variety of reasons. First, it was 
necessary to solidify the administration’s commitment to the project and the changes in the 
health curriculum, which involved the superintendent and both high school principals. Second, it 
was necessary to train and support the participating health teachers to implement the curriculum, 
which included several trainings that supported the teachers’ own mindfulness practices and 
videotaping all lessons across teachers’ classes during all phases of the project to ensure quality 
instruction and fidelity to the program. Trainings were provided both in preparation for 
implementation and to promote sustainability of the mindfulness curriculum in the school 
curriculum. Third, a team visited to administer the quantitative student self-report measures and 
cognitive tasks both prior to the students’ participation in the L2B lessons and after they finished 
the program. Fourth, the qualitative team had to recruit and arrange student interviews. Finally, 
occasional check-ins with the administrators and weekly coaching calls with the participating 
teachers were important to maintain the integrity of the relationship. Given the intensity of time 
required in the schools, it was important to have sites that were within a reasonable distance from 
the university. 


The selection of the sites occurred through pre-existing contacts between both Trish and Mark. 
As the creator of the curriculum, Trish provides trainings nationally and internationally on how to 
facilitate the L2B curriculum. Around the time the grant was being proposed, Trish solicited 
applications to participate in a scholarship-supported training of L2B. All applicants were required 
to have a letter from their principal stating that the participant would actually teach L2B in the 
school, in some form, if they attended. The announcement was circulated through a mindfulness 
listserve. One of the teachers who attended Trish’s training was very interested in offering the 


Page 5 of 19 Evaluating Student Cognitive and Social-Emotional Growth During a 
High School Mindfulness Course Using Mixed-Method Design 


SAGE SAGE Research Methods Cases Part 2 
©2018 SAGE Publications, Ltd.. All Rights Reserved. 


L2B curriculum in her health classes. Mark also had been in contact with the district where the 
schools were ultimately selected for other research projects, so there was a positive prior 
relationship. Given the diversity of the district, the interest of a teacher who had received some 
training, and a positive relationship with the administration, the district was a good fit. 


The trained teacher encouraged another teacher at the other district high school to participate. 
Trish also presented a short program to the district’s health department to gauge additional 
interest. Although it seemed more teachers were interested in getting involved, the time required 
for teacher training and then the abbreviated time for their regular health curriculum when 
implementing the L2B lessons seemed to deter other teachers from participating in implementing 
the L2B curriculum. However, we were able to recruit other teachers at both schools to participate 
in the research by being a part of the “business as usual” control group. This involved allowing 
the team to collect pre- and post- quantitative self-report measures and cognitive tasks around 
the same time as the intervention teachers. 


Clearly, it is important to select research sites that meet the goals of the research. We were very 
fortunate because the two schools where we conducted the research not only fell within the 
same district, but also had some interesting demographic differences. One of the schools was 
more affluent and less racially/ethnically diverse, while the other was more diverse and less 
affluent. This allowed us to examine different cultural and contextual influences. 


It is also crucial to work with flexible and collaborative partners. Fortunately, the teachers were 
very helpful in navigating practical issues and tensions of school research. For example, they 
helped the team figure out how to combine the realities of course schedules with ideal quasi- 
experimental design. In fact, the collegial relationships that developed between teachers and 
research team members strengthened the research process. For the qualitative team, the 
teachers were extremely helpful for gaining insight into the subtlety of the implementation and 
their classroom dynamics, and providing the contextual descriptions that are so important to 
qualitative research. 


Ethical Clearance 


Receiving approval from the university ethics board—and maintaining that approval when making 
changes and updates to the protocol—was a time intensive process. Joy Mitra worked closely 
with the Institutional Review Board of Penn State, whose representatives were very helpful, but 
also rigorous and stringent on ensuring that the research process conformed to the strict ethical 
standards of the university and the protection of minors. 


The IRB staff members were particularly concerned about ensuring that the privacy of the 
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students was protected. Also, as the students were all minors, special protections are in place to 
ensure that students were not coerced into participating in the research. This includes the kind of 
incentives students can receive for participating in the research. For example, the IRB staff did 
not want to have minors receive cash payments as an incentive. Instead, small items, like water 
bottles, gift cards, or backpacks were deemed acceptable incentives. 


The primary lesson from Joy’s IRB work is that establishing a collaborative relationship with the 
individual responsible for ethical clearance is extremely useful as she or he can have useful time- 
saving tips. Also, for research that requires certain changes throughout the research process 
(which most multi-year research does, in our experience) the IRB staff member can facilitate 
those changes and provide advice about when those changes require a formal modification and 
when they do not. 


Each data collection approach needed to be approved by the IRB board, from the survey 
instruments, to the cognitive “games,” to the interview protocols. Since the students were under 
18 years old, the collection instruments also required consent from parents, which was a difficult 
challenge, considering that this required getting the consent letter home to all the parents of the 
students involved in the L2B program and the students in the control group who were taking the 
surveys. The IRB and school district approved a protocol where parents responded with “passive 
consent,” which was very helpful. Passive consent means parents return the form if they do not 
want their child to participate in the research. If we did not receive a reply from parents, then 
students were able to participate. In addition, students provided their assent to participate on the 
first page of the online surveys students completed during their health class. The subsample of 
students who were interviewed had to provide additional consent. Their parents had to provide 
active consent, which involved returning a consent form stating their agreement that their child 
could participate in the interview, and the students had to provide their consent. Because it was 
difficult to achieve active consent to complete the interviews, we offered gift cards as incentives 
for participating. 


Research Design 


We used a quasi-experimental, (intervention and control group with pre and post tests), mixed- 
methods (concurrent triangulation) design to study the effects of the intervention (Creswell, 2013; 
Creswell & Clark, 2007; Creswell, et al., 2003; Maxwell & Loomis, 2003). The research consisted 
of three phases. In Phase 1, the team worked with one teacher to implement the L2B curriculum 
and test out the lessons, the video recording process, and the coaching protocol. Phase 2 was 
the pilot research, where we implemented all of the data collection instruments on multiple L2B 
classes, with two teachers, but we did not collect data from control classrooms. We then 
amended certain data collection practices to ensure reliable implementation in Phase 3: the 
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controlled trial. 


In the Phase 3 controlled trial, the teachers implemented the L2B lessons in their health classes 
while other health teachers at each school taught the standard district-wide health curriculum. 
The students of all classes took the refined battery of surveys and cognitive “games” about 2.5 
months apart, so that measurement occurred before the intervention group began the L2B 
program and after they finished (see Web Resources below, for a sample list of published 
surveys used for this project). During Phase 2, students in the intervention group were 
interviewed after participating in L2B, while in Phase 3 students in the intervention group were 
interviewed both before and after participating in L2B. In the mixed-methods research, we use 
data from both Phase 2 and Phase 3. 


Although the design is fairly straightforward we had a number of difficulties in getting all students 
to take the surveys seriously. Joy reported that some of the students would simply click through 
the questions too quickly to have been reading the survey items, and there was an incident 
where a few students got up and caused some ruckus in the classroom while the students were 
filling out the surveys. The researchers could not help this, but we did take note of the possibility 
of erroneous data. 


In part, the issue of student attention occurred because we underestimated the time 
commitments—and accompanying frustrations—of filling out surveys. Students and teachers 
face constant interruptions of instructional time by State-mandated tests, Advanced Placement 
exams, College Boards, and so on. The teachers are always working upstream to stick to 
curriculum, and the students sometimes feel overwhelmed by the amount of time they spend 
answering multiple-choice questions on the standardized tests that they are required to take. 
Given this context in the classroom, our 2-day data collection activities (which for controlled trial 
classes occupied six full classes of instructional time) sometimes led to frustration and resistance 
on the part of students (and even sometimes for the teachers, too). This is an important lesson 
learned in the data collection phase for school-based research. Schools are very busy places, 
more so now than ever, and researchers need to have high sensitivity to teachers’ and students’ 
time. Knowledge about time constraints is best understood through discussion with teachers and 
with students. 


The qualitative team also had to recruit a subsample of students to volunteer for the interviews. 
We decided that we would go to the schools and offer students some incentive to come talk to us 
about the research project and to ask if they would be willing to volunteer. We started this 
process by asking the teacher to tell the students in the L2B health course that we would have 
pizza and soda for them to learn about the qualitative aspect of the research project. Once we all 
had pizza, we asked the students if they wanted to volunteer to be interviewed. We did this on 
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two separate occasions. We arrived at the school a bit before the students’ series of lunch breaks 
in order to set up the pizza and sodas. We had a number of students come in during each 
period. 


Convincing adolescents to do something that may be outside of their immediate interests is never 
an easy endeavor. Fortunately, each person on the qualitative research team had been a K-12 
teacher, and one had been a principal, so we put on our “teacher hats” to explain to the kids who 
we were, what we were doing, and why we needed their help. We also told the students that if 
they completed the interviews, they would receive a US$15 gift card. We told the students that 
they needed to have their parents sign a consent form, and then we would text them about a 
time to meet either at the school or by phone. We were careful not to be coercive in any way, and 
follow the proper IRB guidelines. 


Some students were more interested than others. We had over 30 students say that they were 
willing to participate in interviews, but only 19 students ended up getting the paper work signed 
and completing the interviews. This was a little disappointing to the qualitative team as it limited 
the diversity of students from whom we were hoping to collect qualitative data. However, we were 
still able to complete our research objectives. 


Getting adolescents to volunteer to do anything, even with the promise of some kind of reward, is 
a difficult aspect of school-based research. Although we were all teachers and knew the age 
group, there are characteristics of the teenage stage of development that just cannot be 
controlled for—mostly that they are wrapped up in their own lives. Without a significant 
investment or a major incentive (we offered a minor incentive), many teenagers are unlikely to 
follow-through on their tertiary, extra curricular commitments, and those who do, often need 
constant reminders. 


Recruiting volunteers for this study had a significant methodological drawback. We were unable 
to achieve maximum variation sampling (Patton, 2005), which would have provided a more 
comprehensive set of findings for the qualitative aspects of the research. The students who were 
willing to be interviewed were generally responsible, moderately to high achieving teenagers. 
Nonetheless, we did achieve some diversity in the sample. Furthermore, the qualitative data 
allowed us to triangulate the quantitative findings and add complexity and nuance to the 
outcomes report of the intervention. Before we discuss those outcomes, however, we need to go 
a little more in depth on the data collection. 


Data Collection Design 


We collected quantitative data and qualitative data concurrently to triangulate the findings 
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(Creswell, et al., 2003). This means that we collected the quantitative data in a separate process 
from qualitative data even though both sets of data were collected during the same time period. 
To perform this task, the quantitative team developed their protocols separately, though with input 
by the qualitative team, and the qualitative team developed their protocols separately, but with 
input from the quantitative team as well. 


An expert in their particular methodology led each team. It was an interesting experience to delve 
into the different epistemologies, or approaches to knowledge and gathering data, that each 
method asserted when creating each protocol. It was tricky to reconcile these different 
approaches because the quantitative team approached their survey selection and creation 
through finding carefully worded, direct questions that measured the outcomes of interest and 
tested their hypotheses. So, the items were tested for validity and reliability. This means that the 
quantitative team made sure that the questions were validly measuring the right ideas (that the 
stress survey was actually measuring stress) and that they were reliable (meaning that the stress 
survey was accurately measuring the amount of stress) (Fink & Litwin, 1995). This approach in 
some ways runs contrary to the inductive approach that the qualitative team wished to employ for 
the student interviews. We sought to understand how the students made sense of and 
experienced the L2B course, in their own conceptualizations, and any changes they may have 
noticed within themselves because of the course. The qualitative team spent considerable time 
creating the open-ended, constructively neutral questions to allow students to find the words that 
made sense to them in order to understand their experience from their perspectives as much as 
possible, while still focusing on the topic of the mindfulness course. 


When the qualitative team showed the interview protocol to the quantitative team for feedback 
there was some confusion about the approach, and why the questions were so broad. We 
explained that one of the benefits of qualitative interviews is the opportunity to delve deeply into 
the contextual, experiential knowledge from the students’ perspectives, in their own words. We 
wanted to know what they found important and useful instead of imposing certain ideas on them. 
In addition, our experience as qualitative researchers has helped us realize that when individuals 
are asked directly how something has impacted them, they frequently either do not know or they 
offer responses lacking in clarity. However, when those same individuals are asked to describe a 
situation or tell a story, their responses provide rich reflections about the same phenomenon. 
Because we wanted to conduct a truly mixed-methods study we also wanted to develop some 
questions that might correspond across methodologies. As a result, at the end of the interview 
we incorporated some more direct questions about the programming and certain constructs in 
order to make sure all topics were covered directly. 


Another aspect of data collection design is to develop analytical approaches to understand the 
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data collected. For the quantitative team, their deductive approach to data gathering ensured that 
the analytical approaches, such as analysis of variance (ANOVA) and Latent Class Modeling, 
were already part of the data collection design. For the qualitative team, we needed to develop a 
codebook that would allow us to code the data and find themes within the data relevant to our 
investigation. In qualitative research, developing code-schemes can happen both a priori (before 
the data is collected) and a posteriori (after data are collected). We decided on a hybrid 
approach where we developed codes for specific themes that we wanted to understand. For 
example, we used some of the quantitative indicators, such as mindfulness and stress, as codes. 
However, we also allowed ourselves the flexibility to find emergent, or unanticipated themes 
coming out of the interviews. This allowed us to uncover important data that contributed to our 
understanding of the influence of the L2B program, which we had not anticipated. 


Methods in Action 


Out of the many strengths and challenges within mixed-method research the most salient for our 
particular research project was how to incorporate the information gathered from each method to 
create an accurate understanding of how each individual experienced stress in their lives and 
how the L2B program impacted them. We decided to utilize the qualitative data for two main 
purposes: (1) contextual information about each student’s unique situation and (2) explanatory 
information about each student’s unique experience with the L2B program and how they were 
impacted in different ways. 


To achieve this, we first had to figure out what outcomes and relevant constructs were covered by 
both methods. Because the qualitative work was more inductive and emergent, and the 
quantitative data were more deductive and structured, we decided to use particular constructs 
from the quantitative data that overlapped with the emergent themes that came out of the 
qualitative data. We settled on 10 themes that overlapped. Table 1 below demonstrates how we 
were able to bridge constructs: 


Table 1. Construct Bridge. 


Quantitative constructs Qualitative constructs 


Self-compassion Emotional well-being 


Internalizing behaviors—depression, rumination, ; : 
Emotional well-being 


anxiety 
Attention—mindfulness, mind wandering Mindfulness 
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Physical symptoms—sleep, somatization Physical well-being 


Stress — school, peers Stress 


Interpersonal interactions, 
Relationships — social connectedness 
belongingness 


Emotion regulation Mindfulness (nonreactivity) 


Externalizing behaviors—substance use, ; ; 
Physical well-being 
consequences 


Academics 


Efficacy 


After significant discussion, we settled on an approach to combine the two sets of data. From the 
student surveys, the quantitative team already had the aggregated results of the pre- and post- 
intervention outcome measures. We decided to utilize the quantitative data for the 10 students 
we interviewed and compare them individually to the whole sample. Once we had this 
information, Deborah, who headed the qualitative team, created a “quantitative profile” describing 
each student along the eight constructs from data analyzed by the quantitative team. This 
quantitative profile essentially was a short descriptive memo about each student based on the 
quantitative outcomes and how they changed or didn’t change as a result of participation in the 
mindfulness class. The students from Phase 3 also completed an additional round of data 
collection, with follow-up data taken 5 months after they completed the L2B program. 


For the qualitative analysis, Julia and Joe created descriptive memos for each student using only 
the qualitative data to describe their profile based on the 10 qualitative constructs. We selected 
relevant quotes from the interview transcript, and performed a content analysis to examine what 
the student said about each construct. We then constructed a “qualitative profile” that interpreted 
student quotes. 


Understanding the Findings 


Once we completed the profiles separately, we then brought the quantitative and qualitative data 
together to see what we could learn from the comparison. We should note that the quantitative 
and qualitative profiles were written independently, meaning Deborah did not refer to any 
qualitative data while completing the quantitative profiles, and Julia and Joe did not refer to any 
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quantitative data while completing the qualitative profiles. The team did this purposefully as we 
felt it was important to create a profile for each student based only on one methodological 
paradigm to get a full sense of what each set of data “said.” When the outcomes were compared 
across methodologies, many findings made sense, but some of them were confusing, though 
ultimately helpful. Below are two examples about how the quantitative and qualitative data 
worked together to provide triangulation for cases of divergent data “stories,” as well as what that 
triangulation looked like and meant for this work. 


Example 1 


Javier, (a pseudonym), had a surprising finding when it came to mindfulness. Compared to the 
pre-test, both his mindfulness and mind-wandering scores went up. We expected that if 
mindfulness scores went up, then the mind-wandering scores would go down. Because this did 
not happen, we were curious to understand why. In the qualitative analysis, Javier talked 
specifically about his difficulty staying focused, but that he was now more aware of his mind- 
wandering as a result of learning mindfulness techniques, which explained the surprising 
outcomes. 


Example 2 


Hannah (also a pseudonym), scored lower than the average of her peers on social 
connectedness in her survey responses post course, and she scored relatively higher on 
rumination post intervention as well, (which is the opposite of the desired outcome). However, 
during her interview, she said that she relied upon her friends to relieve stress, and talked 
positively about her friends. She also mentioned that she lost her grandmother a few weeks 
before the semester, which was very challenging for her and for her mother, who she ended up 
needing to support. She said that this was the most challenging and stressful experience of her 
life, and that it was isolating. However, she said that the L2B course was helpful for her to 
become aware of these feelings and reduce her stress. This explained why, although all other 
indicators showed her to be a healthy young person, and that the L2B course had a positive 
effect on her wellbeing, certain outcomes showed negative results. 


Finding meaning in divergent outcomes, such as in these examples, was a complicated process 
for the mixed-method design we utilized. For example, it is difficult to know how the averages of 
the student outcomes were affected by these individual circumstances. In theory, individual 
circumstances would affect both control and intervention groups equally, so this kind of 
background would be controlled for by the quasi-experimental design from a group analysis 
standpoint. However, contextual circumstances are still experienced differently by each student 
and the mindfulness class may have affected how they responded to a stressful life event. 
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Though challenging, we found important information from this mixed-method approach that is 
unique to the use of a coordinated approach using both methodologies. For example, Javier and 
other participants offer evidence that there are two ways in which the mindfulness training could 
have supported students’ growth. The practices might help youth increase their mindfulness and 
lower their mind wandering, or alternatively, the practices might increase students’ awareness 
about their mind wandering, which indicates greater self-awareness, and potentially far less mind 
wandering in the future. The mixed-methods approach uncovered this potentiality, which would 
not have been discovered with only a quantitative or a qualitative approach. In a quantitative 
approach, there would not have been explanation, personal insights, and context offered for 
these findings. For the qualitative methods, the outcomes would not have been as clear cut or 
easy to understand in terms of larger populations if we had only interviewed students. These 
findings will be able to contribute to more in-depth theoretical constructs about the relationship 
between mindfulness practices and wellbeing measures, which is why this methodology offers 
useful insights, even if it can be more conceptually and practically challenging than single 
method approaches. 


Practical Lessons Learned 


Based on the discussion so far, we learned some practical lessons about quasi-experimental, 
concurrent triangulation, mixed-methodologies for psychological research: 


1.Ensure you perform both methods in-depth. The quality of the findings would not have been 
nearly as robust if only the quantitative team or the qualitative team attempted to collect and 
analyze the data using both methods on their own. There is a value to methodological 
specialization, as long as both teams are able to be open and also learn and value others’ 
methodological approaches. 

2.Work in groups. It is helpful to have a large team to perform concurrent triangulation for a 
quasi-experimental research project, so that the data collected can be in-depth and high 
quality for both methods. It is also helpful to have team members who possess an openness 
and appreciation for both approaches. There are some challenges with large teams, however, 
such as communication and the ability to integrate the different epistemological frameworks 
that each orientation offers. Patience, a willingness to learn, and good humor are the best 
tools for ensuring all ends up well. 

3.Find a balance between autonomy and communication. Autonomy, combined with a lot of 
communication is a valuable but complicated endeavor. In this case, it lead to some tensions 
among the group to get everyone on the same page, but in the end, it was worth it. 

4.Strong interpersonal skills are helpful for gaining access to a school research site. Constant 
communication and mobilizing various social networks are important for gaining access to 
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schools. It is not easy to convince people within a school to agree to participate in a research 
study, but some incentives for the teachers are often helpful, if you have the money. It is also 
important to build trust and ensure that there is a two-way relationship between the school 
and the researchers. 

5.Work closely with your institution, if possible, for ethical review. Institutional review of ethics 
for school research is a long and tedious process, but it pays to be friendly with your review 
specialist so that changes to the protocol can be facilitated more easily. 

6.Test your tools. Testing and discussing your research tools are essential for quality data 
collection. Although this was not discussed in the case specifically, it is important to 
remember. 

7.Understand the developmental stage of your research population. Without the proper 
motivation and communication techniques, it might have not been possible to convince 
teenage students to engage in qualitative research, and even with that knowledge it was 
difficult. It is important to know the developmental stage and the related approaches for 
working with your research population. 

8.Overestimate how much time it will take to collect data. Anticipate the issue of limited time for 
data collection, and look for the best balance between the time requirements for data 
collection and the value of the data collected. Teacher enthusiasm for working with a cutting- 
edge curriculum does not extend to providing so much time for data collection. Additional 
upfront time at study initiation to build support and understanding for the goals of the study 
might be time well spent, as well as discussions about navigating time commitments with 
teachers and students. 


Conclusion 


This case study demonstrates the value, as well as some of the difficulties and lessons learned, 
in a mixed-methods approach for a quasi-experimental design research project in a high school 
context. This mixed-method approach allowed our research team to uncover useful information 
that would not be understood with only one method. Although there was a significant amount of 
discussion and it was difficult to reconcile the different approaches, ultimately we were able to 
find a useful synthesis for the design. The outcomes of the qualitative—quantitative comparison 
offered us a more holistic picture of the learning and changes that students underwent in a high 
school health course that incorporated mindfulness. 


Exercises and Discussion Questions 


1.What are the strengths and weaknesses of working in a large research team? What might be 
some ways to mitigate the disadvantages? 
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2.We called the design of this study quasi-experimental, concurrent triangulation, and mixed 
method. What do each of these terms entail for the research design, that is, what do each of 
the terms mean? What are other approaches that might get at the same data and analysis? 

3.Because the research was performed in a similar cultural context to the background of the 
researchers’, in this case study, we did not discuss context and culture very much. However, 
culture and context are essential considerations for any research project. Do you think this 
study could be performed in cultural contexts different from a public high school in an 
Eastern U.S. state? If so, why? If not, why not? 

4.How did this study navigate the differences between quantitative and qualitative 
epistemologies? How else might researchers navigate these differences? 

5.What are some of the drawbacks for this particular research design? What are some of its 
unique strengths? 

6.Why are interpersonal skills so important in this kind of research project? 

7.1f you were to perform a research project similar to this case, what design elements would 
you keep, and which would you change? 

8.Writing prompt: Create your own mixed-method, quasi-experimental research plan to test an 
educational interventions’ effect on students. Take into consideration the lessons learned from 
the case. 


Further Reading 


Davidson, R. J., & Kaszniak, A. W. (2015). Conceptual and methodological issues in research 
on mindfulness and meditation. American Psychologist, 70, 581-592. 


Metz, S. M., Frank, J. L., Riebel, D., Cantrell, T., Sanders, R., & Broderick, P. C. (2013). The 
effectiveness of the Learning to BREATHE Program on adolescent emotion regulation. Research 
in Human Development, 10, 252-272. doi:http://dx.doi.org/10.1080/15427609.2013.818488 


Web Resources 
Learning to Breathehttp://learning2breathe.org/ 


Self Compassion scale —Short form, 12 items (SCS-SF, Raes et al., 2011) https://self- 
compassion.org/wp-content/uploads/2015/02/ShortSCS. pdf 


Primary Health Questionnaire, 8 items (PHQ, Spitzer et al., 
1999)http://www.phqscreeners.com/select-screener/1 11 


The Rumination and Reflection Questionnaire (RRQ, Trapnell & Campbell, 1999) 
http:/Awww.mindfulness-extended.nl/content3/wp-content/uploads/2013/07/RRQ-Rumination- 
Reflection-Questionnaire1 .pdf 
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Generalized Anxiety Disorder, 7 items (GAD, Spitzer, Williams, & Lowe, 
2006)http://www.integration.samhsa.gov/clinical-practice/GAD708.19.08Cartwright.pdf 


Child and Adolescent Mindfulness Measure (CAMM-SF, Greco, Baer & Smith, 2011) 
http://www.ruthbaer.com/academics/CAMM. pdf 


Difficulties in Emotion Regulation Scale (DERS, Gratz &, Roemer, 
2004)http://cairncenter.com/forms/difficultiesinemotionalregulation_scale.pdf 


Social Connectedness-Revised (SCC-R, Lee, Draper, & Lee, 
2001)http://depts.washington.edu/uwcssc/sites/default/files//Social%20Connectedness%20Scale- 
Revised.pdf 
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